
Teaching Formative Assessment 
Strategies to Preservice Teachers: 
Exploring the Use of Handheld 
Computing to Facilitate the Action 
Research Process 

Kristin Redington Bennett and Ann C. Cunningham 


Abstract 


Appropriate classroom assessment now tends to utilize formative measures 
with greater frequency, especially in the early grades and with learner 
groups at risk of not passing state-mandated standardized tests. Within 
the authentic context of an action research project, teacher candidates 
were given handheld computers equipped with data-collection software 
to assess the effectiveness of tutoring sessions with students identified with 
special needs. The data was collected and reviewed weekly as formative 
assessment and was also analyzed over time for performance trends. The 
goal was to explore whether the introduction of handheld data collection 
tools and new pedagogical practices embedded in an action research project 
is a feasible expectation for novice teachers. Results from postexperience 
questionnaires and analysis of recurrent themes in written refections 
indicate that the teacher candidates valued the action research and 
formative assessment process. Although the hardware created a variety of 
challenges to data collection, all participants recognized the value of the 
handheld computer for classroom-based formative assessment. This study 
is the first step in a long-term research study on the efficacy of handheld 
tools to support formative assessment in the elementary classroom. (Key- 
words: formative assessment, action research, preservice teachers, handheld 
computers, one-to-one computing) 


Introduction 

T he effectiveness of any instructional program often turns in part on 
the use of effective assessment materials and tools. Although teach- 
ers strive to create meaningful learning situations involving the “whole 
child,” they often rely on summative, standardized assessment strategies, 
which are a poor fit to instructional programs (Means, Penuel, and 
Quellmalz, 2001). Improving classroom assessment practices has proven 
to he challenging. Assessment, especially formative assessment, is given 
little attention in classrooms, where teachers rarely have adequate time 
to plan or implement assessment activities in a methodical manner or 
to learn new strategies for assessment from peers or experts (Black & 
William, 1998; Darling-Hammond, Ancess, & Falk, 1995). Further, 
research suggests that even those teachers who do engage in some form of 
formative assessment are more apt to look at individual behaviors rather 
than systematically assessing student learning ( Mandinach et al, 2006; 
Mandinach et al, 2005; Confrey & Makar, 2002, 2005) 

Student performance expectations established by state education agen- 
cies in response to No Child Left Behind (NCLB) legislation provide 
an impetus for a greater focus on formative assessment in the classroom. 
Assessment must be viewed as an integral part of instruction because it 
“not only provides information about learning, it drives learning” (Lewis, 
2006, p. 29). Well-designed formative assessment instruments that 
capture performance data on individual students are a vital component 
of student success on formal benchmark assessments, and interim assess- 
ment is necessary to diagnose student needs for additional instruction 


(Lewis, 2006, p. 17). What is needed is a method that teachers can easily 
employ in the classroom to make formative assessment timely, “provide 
quality feedback, strengthen teaching and learning, and involve students 
in accountability” (Lewis, 2006, p. 30). Although experts in the field of 
psychometrics report that this type of assessment practice is valuable, they 
also admit that it is not currently being implemented in K- 12 classrooms. 
The Center for Research on Evaluation, Standards, and Student Testing 
(CRESST) Co-Director Eva Herman states, “These types of assessments 
are often nonexistent in curriculum materials, in teachers’ repertoires, and 
not often found in benchmark tests. According to Lewis (2006), teachers 
do not have time to devise these types of assessments on their own (p. 
29). A need exists to increase knowledge and understanding of formative 
assessment techniques in teacher preparation programs. 

This paper describes the results of a preliminary exploration of the 
integration of an action research model that emphasizes the role of forma- 
tive assessment using data collection with handheld computers. The goal 
is to scaffold the development of sound instructional decision-making 
practices by teacher candidates during field experiences in elementary 
classrooms. The purpose of this study was to determine the efficacy of 
such a model with novice teachers, and the focus of this paper is the use 
of a handheld computing device to facilitate collection of formative as- 
sessment data in the field. 

The study explores the following questions: 

1 . Do teacher candidates find handheld computers useful for con- 
ducting regular formative assessment? 

2 . Do preservice teachers recognize the value of formative assessment 
as a method for improving instructional practice? 

Literature Review 

Action Research 

This research project is framed within the contexts of action research and 
situated learning. Herrington and Oliver (1999) compiled a list of “nine 
characteristics of a situated learning framework, namely: an authentic 
context; complex authentic activities; multiple perspectives; expert per- 
formances; coaching and scaffolding; opportunities for collaboration, 
reflection, and articulation; and authentic assessment” (p. 402). 

The framework provided by that of situated learning complements 
the methodological framework of action research. Cohen and Manion 
(1989) suggest that action research can be difficult to define; however, 
they also assert that the research literature reveals this method to have 
tangible features: 

Action research is situational — it is concerned with diag- 
nosing a problem in a specific context and attempting to 
solve it in that context. It is usually collaborative — teams 
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Figure 1: Model of Action Research Process Supported by Handheld Data 
Collection, 


of researchers and practitioners work together on a proj- 
ect; it is participatory — team members themselves take 
part directly or indirectly in implementing the research; 
and it is self-evaluative — modifications are continuously 
evaluated within the ongoing situation (p. 217). 

Action research is often represented as a four-phase cycli- 
cal process where a plan is formulated, that plan is acted 
upon, the outcomes are observed, and reflection is under- 
taken to understand the processes, strengths, and weak- 
nesses of the effort. The conscious analysis or reflection 
stage of action research distinguishes it from the casual 
“plan, act, sense, and re-plan” cycle by which we operate 
in everyday life (Tripp, 1990). 

Recent studies have asserted that undergoing action research as part of 
their education programs can benefit undergraduate preservice teachers 
(Price, 2001; Rock & Levin, 2002). As the benefits of action research being 
demonstrated in these studies and other studies (Burnaford and Hobson, 
1995; Johnson and Button, 2000; Sax and Fisher, 2001) of graduate-level 
teacher education students, Fueyo and Koorland (1997) have called for 
action research to be included as part of preservice education programs. 
Using action research strengthens professional development opportuni- 
ties (Mills, 2003) and has strong relevance for preservice and inservice 
teachers, especially as related to increasing student performance through 
reflective practice enriched by the collection of data that facilitates a 
deeper understanding of student performance. 

Formative Assessment 

Formative assessment in and of itself is a form of action research. Whereas 
action research is generally done in a learning context by teachers as 
professional development, formative assessment is done in a teaching 
context and is a natural component of action research. In Mathematics 
Assessment Probes, Rose et al. (2007, p. 14) suggest that formative assess- 
ment should include working through a QUEST cycle (see Figure 1), 
which includes: 

• Questioning student understanding of a particular concept 

• Uncovering understanding and misunderstandings using a 
problem 

• Examining student work 

• Seeking links to cognitive research to drive next steps in instruc- 
tion 

• Teaching implications based on findings and determining impact 
on learning by asking an additional question. 


This systematic cyclical approach eschews the conventional single- 
purpose, cumulative, high-stakes accountability testing in favor of 
data-driven instructional decisions based on assessment for learning. As 
Stiggins (2005) suggests, one reason for the recent resurgence of interest 
in formative assessment has been educators’ realization that once-a-year 
summative standardized testing doesn’t happen frequently enough to 
affect specific day-to-day, week-to-week, or even month-to-month in- 
structional decisions. Further, such testing fails to provide a sufficiently 
detailed picture of student learning to enable teachers to identify ways to 
help individual students. As teachers are recognizing the value of data- 
driven instructional models, a natural next step is to establish systematic 
methods and applications for collecting and analyzing this data in a 
timely, day-to-day manner. 

Handhelds 

Handheld computers offer the opportunity to facilitate teachers’ imple- 
mentation of continuous, authentic, and reflective assessment in the 
classroom. Sheilds and Poftak (2002) suggest that “handhelds offer 
portability while at the same time packing enough computing power for 
writing, crunching numbers, and surfing the Web. . . . infrared beaming 
offer opportunities for real-time student collaboration, and with pricing 
near half the cost of a laptop, the days of one computer for every student 
may be tantalizingly close at hand” (p. 24). Because of their portability, 
handheld computers ease the data-gathering process because of their 
unobtrusiveness and ease the aggregation of data because of their tech- 
nological features. Many researchers have heralded the myriad potential 
uses of handhelds in held investigations (Hsi 2000; Soloway et al., 1999). 
Ongoing, in-the-moment, formative assessment made easier by the por- 
tability of handhelds means that assessment can occur in any location at 
any phase of the learning process. Further, their affordability means that 
they are a feasible option for widespread use in our schools. 

Developments in pedagogy and learning science have moved toward 
processes that engage a more active learner using constructivist models, 
with learners making their own decisions that match their cognitive needs. 
Involving preservice teachers in the learning of formative assessment pro- 
cesses using handhelds devices certainly lends to an active, constructivist 
learning experience. 

Research Methods 

To determine the value of handheld computing devices for the collec- 
tion of formative assessment data in elementary classrooms, the author 
implemented an action research project with two groups of preservice 
teachers at a small, private university in the southeast United States. The 
handheld computing tools were used iPAQs owned by the university and 
were available at no charge to the student participants, the data-collection 
software was also a university creation that was available for free, and 
on-site technical support was readily available. Despite the age of the 
equipment, it was determined that these free tools would be appropriate 
for investigation of the application of a handheld for data collection in 
the held. The researchers were aware of the future direction of handheld 
computing devices and wanted to explore the efficacy of such tools within 
a complex pedagogical model prior to committing to the extensive costs 
of state-of-the-art handheld computing solutions. Therefore, this paper 
focuses on questions relating to the use of a handheld computing device to 
facilitate formative assessment data collection and to scaffold the develop- 
ment of data-driven instructional practice habits in preservice teachers. 

Answering the questions regarding the utility and teacher-perceived 
effectiveness of using handhelds to facilitate the teaching and learning 
of formative assessment practices was done through a complex model 
that embeds on a dual level the QUEST (Rose et al., 2007) diagnostic 
strategy of formative assessment within the action research process. The 
following discussion of the research process will describe this process of 
situated learning for preservice teachers. 
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Data Sources and Analysis 

The participants were all teacher candidates in the final semester of their 
elementary education program. Data was collected with two groups of 
participants with similar backgrounds during the spring semester of two 
consecutive years. These participants were a convenience sample (Group 
1 n=17, Group 2 n=15) who were all enrolled in a required course, 
“Teaching Ghildren with Special Needs,” the final class in the elementary 
education course sequence. This course includes a field experience that 
requires the participants to he in a local elementary school tutoring the 
same student every week. All participants successfully completed their stu- 
dent teaching experience in the prior semester, and the participants all had 
extensive technology experience in part due to the ubiquitous resources 
of a one-to-one laptop university and through their teacher education 
coursework, which includes a required technology course. Several of the 
participants had prior experience using handheld computing devices, but 
not for collection and analysis of data on student performance. 

The use of handheld computers to facilitate data collection was made 
for several reasons; one primary criterion was related to the availability 
of the hardware and software for no cost to the students. The research- 
ers’ knowledge of the future direction of handheld tools encouraged the 
exploration at the beginning of the research project even though the tools 
were not state of the art. The decision was made to use older hardware 
and study the efficacy or perceived efficacy of handheld computing for 
data collection and determine whether novice teachers recognized the 
value of formative assessment within an authentic teaching context. The 
integration of a data-collection tool that allows users to create surveys on 
their laptops and transfer the survey easily to the handheld and the results 
easily to the computer was an innovation important for future teachers 
to experience, especially within an authentic context. 

The researchers issued both groups of participants their own iPAQ 
handheld devices with associated accessories and portable keyboards. They 
received training on the handheld through a collaborative training session 
designed by members of the Information Systems Department and faculty 
members from the Department of Education. The first training session 
included an introduction to the handhelds and their general features, as 
well as features specifically designed for students at the university. The 
first group of participants was given time to experience (play with) the 
handhelds over a period of two weeks before being introduced to the 
field data-collection software, DataInHand (http://datainhand.wfu. 
edu), a simple survey-creation software designed for handheld devices. 
DataInHand was developed by the research and development team in 
the Information Systems division at the university. The features of this 
software include a simple interface that permits survey development on 
the computer (Likert scale, yes/no, short-response questions) and simple 
transfer and retrieval of completed surveys to and from the handheld de- 
vice. Results can be reported in HTML, Microsoft Word, and Microsoft 
Excel formats. 

Process and Implementation: Year One 

To more strongly situate the action research process, we asked participants 
to select one student identified with special needs from their student 
teaching experience to tutor one on one for 2 hours each week. We asked 
participants to work with school personnel to identify learning challenges 
and establish growth goals for the chosen student, and we guided them to 
address “the whole child” in their tutoring and include assessment items 
that related to affect and behavior in addition to learning goals. After con- 
ducting the needs assessment and identifying learning goals for the tutee, 
we asked the participants to design an assessment instrument to measure 
the effectiveness of their instructional interactions with the student as well 
as any affective or cognitive changes they noted during their sessions. The 
researchers provided training on the data-collection software that included 


coverage of the software features, survey creation, data collection, and 
transfer of data-collection instruments and data between the handheld 
and the laptop. The researchers involved with this project as well as sup- 
port personnel from the university’s Information Systems division were 
made available to answer questions about hardware or software during 
this time, but the content of the instrument was largely determined by 
the participants, as they knew the students they would be tutoring from 
the previous semester as student teachers. The researchers then asked the 
participants to perform several trial runs of the instrument with fictional 
information to practice the process of downloading, uploading, and 
reporting in an effort to promote efficient data collection and storage. 
After reflection on initial data collection experiences and discussion of 
the results, the participants decided to create an instrument that was 
consistent across all tutoring situations and that allowed individuals to 
add additional questions specific to his/her child’s special needs. Once the 
questions were determined, all participants entered it into the DataIn- 
Hand program while in the university classroom. As part of their action 
research plan, the participants agreed to use this instrument in the field 
for 2 weeks and then come together to reflect on and modify either the 
instrument or the process. 

After the 2-week trial period, the participants determined that the 
instrument needed modification to include fewer open-ended ques- 
tions. At this time, the participants also shared their individually created 
“additional questions’” which led several participants to modify their 
instruments to include questions particularly relevant to their individual 
tutees. The participants then utilized this modified instrument for the 
remainder of the semester, repeating the administration and data collec- 
tion via handheld computer on a weekly basis. Glass sessions included 
guided reflection on the process of data collection and analysis as well as 
discussion of the efficacy of the handheld computers. 

Process and Implementation: YearTwo 

The data collected from the first year of the study lead to small but 
significant modifications in the second year of the study. Review of the 
participants’ project results, postsurveys, and anecdotal evidence identified 
several significant findings that affected the design of the second year of 
the study. These included: 

• Hardware problems often made the technology burdensome 
rather than helpful. 

• Assessment instruments need to be short, and questions must 
permit input via the stylus rather than the keyboard. 

• Although open-ended questions yield rich data, they make field- 
based data collection challenging. 

The researchers determined that the early distribution and training 
on the handhelds at the end of the fall semester was necessary to increase 
the time for participants to familiarize themselves with the hardware and 
reduce the frequency of hardware problems. Additionally, the researchers 
scheduled more opportunities for survey design and modification as well 
as hardware and software support during the regular class sessions. These 
sessions were added based on experience gained during the first year of the 
study and were designed to help overcome hardware and software barriers 
and to provide opportunities for group reflection on the data-collection 
process and redesign of surveys. 

Another significant change included the placement of all partici- 
pants in a new elementary school partnership. None of the participants 
completed their student teaching in this school, and the classroom for 
this field placement was entirely comprised of students identified with 
special needs. 

The change in setting, as well as analysis of the data from year one, led 
to several changes in the instrument content, creation, and data collection. 
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Table 1 : Measures of Central Tendency for Responses to Postexperience Questionnaires of Teacher Candidates Regarding Formative Assessment 
and Use of Handhelds to Support Field-Based Data Collection and Analysis 


Questions 


2006 


2007 


Mean 

Mode 

Mean 

Mode 

1 . The handheld is a useful tool for collecting data in the field. 

2.07 

3 

2.42 

2 

2. The assessment instrument made it easierto capture data in the field. 

2.14 

3 

2.47 

2 

3. The handheld facilitated completion of course assignments. 

2.21 

3 

2.42 

2 

4. The handheld and assessment instrument allowed me to collect and process field observations in a timely manner 

2.21 

2 

2.05 

3 

5. Using a handheld device and a consistent data collection tool facilitated data analysis. 

2.35 

3 

2.36 

3 

6. Using these tools to collect data in the field was worthwhile. 

2.0 

2 

2.11 

2 

7. My understanding of howto use technology for formative assessment has been enhanced by this experience. 

2.64 

3 

2.52 

3 


Participants were provided with an exemplar instrument adapted from 
the year-one study that included fewer open-ended questions and more 
Likert-type items. The instrument items (see Appendix A) included more 
questions on the affect of both the tutor and the tutee during the tutoring 
sessions. Participants were still given the option to include questions that 
specifically related to their tutees. The feedback on challenges associated 
with keyboard-based data entry compelled the researchers to encourage 
question design based on state performance standards and the state as- 
sessment scale, which could be facilitated by Likert-type questions and 
removed the need to enter data with a keyboard. The researchers suggested 
the possibility of including state Standard Course of Study objectives as 
assessment items but left the final decision up to the participants. 

Findings 

This 2-year study generated a wealth of data from the participants regard- 
ing preservice teachers’ awareness of how formative assessment effects 
instructional practice and the value of handheld tools for collecting 
formative assessment data. Each participant completed a postexperi- 
ence questionnaire consisting of Likert-type and open-ended response 
questions that asked them to describe their attitudes about the use and 
value of the handheld device for collecting formative assessment data and 
completing tasks that require field-based data collection. 

The data garnered from the participants’ data collection in the field 
provided information on the weekly successes, setbacks, and other impor- 
tant details about both the participants and their students and was used 
to guide decision making about tutoring as well as to inform their written 
reflections about the action research process and tutoring experience. 

Do Teacher Candidates Find Handheld Computers Useful 
for Conducting Regular Formative Assessment? 

To determine teacher candidates’ attitudes toward using handhelds for 
formative assessment, the researchers asked seven postexperience questions 
that required the participants to rate their response on a scale from 1 to 
4. Table 1 shows the questions from the postexperience questionnaire 
for both years of the study (2006 and 2007) and provides the means 
and modes for responses from each group on a 4-point scale as follows: 
l=strongly disagree, 2=disagree, 3=agree, 4 =strongly agree. 

Responses from participants in both groups were analyzed in measures 
of central tendency to show more complete data for each small group. 
Based on the mean, participants in the 2007 study group agree more 
often than participants in the 2006 study group that the handheld was 
a useful tool for collecting data in the field. In response to question 2, 
participants in the 2007 group agree more often that the software made it 
easier to capture data in the field than respondents in the 2006 group. In 
response to question 3, members of the 2007 group agree more often that 


the handheld facilitated completion of course assignments than members 
of the 2006 group. Although the means for both groups’ responses tend 
more toward disagreement to question 4, the mean for the 2006 group 
is higher and the mode indicates that more members of the 2007 group 
agreed that the tools facilitated processing field observations in a timely 
manner. In response to question 5, the mode indicates that the majority 
of respondents in both groups agreed that when used consistently, the 
tools facilitated data analysis. However, their responses to question 6 
indicate that both groups did not agree that using the tools to collect 
data in the field was worthwhile. Both groups were consistent in their 
responses to question 7, and the mode reveals that their understanding 
of technology use to collect formative assessment data was enhanced by 
this experience. Both measures of central tendency were higher for both 
groups on this question. 

Figure 2 presents a visual representation of the comparison of the 
means for the responses from both the 2006 study group and the 2007 
study group. Figure 3 presents a visual representation of the comparison 
of the modes for the responses from both groups. 

To further cull out the idiosyncrasies of the study groups’ mixed 
responses to the postexperience questions, the researchers looked to can- 
didates’ responses to the following two open-ended questions regarding 
suggested changes: 

1 . What would you recommend we change before implementing 
handheld computers and DataInHand for this academic purpose 
in the future? 

2. Please share any comments about your experience with field ob- 
servations that would help us improve training, data collection, 
or data analysis in the future. 

Content analysis of the combined responses mimics some of the 
mixed reactions found in the quantitative indices. However, the qualita- 
tive data also reveals that candidates recognized the potential benefits for 
using the handhelds devices and were able to separate the problems with 
the hardware, the problems with the software, and the actual context 
for learning formative assessment. The coded data indicates that the 
most suggested change surrounded the candidates’ frustration with the 
hardware. Secondarily, candidates said that using the right software and 
instrumentation were vital aspects of a successful experience. Finally, 
despite the hardware and software issues, the candidates were optimistic 
about the technology and potential usefulness of using handheld devices 
to collect data. Table 2 provides exemplar quotes from each theme that 
emerged during content analysis. 

Although there were mixed feelings about the use of the iPAQ as 
a means for ongoing field-based data collection, the participants were 
able to grasp the value of intentionally recording data to make informed 
instructional decisions, which was an important growth experience for 
novice teachers. 
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Figure 2: Comparison of Means for Both Study Groups' Responses to Post- 
experience Questionnaire. 



Figure 3: Comparison of Modes for Both Study Groups' Responses to Post- 
experience Questionnaire. 


Do Preservice Teachers Recognize the Value of Formative 
Assessment as a Method for Improving Instructional 
Practice? 

By and large, preservice teachers recognized the value of formative assess- 
ment as a method for improving instructional practice. Analysis of the 
responses to the open-ended question, “How did the data inform your 
sessions on a day-to-day basis?” indicate preservice teachers’ discovery 
awareness of the purpose for formative assessment. The following selected 
quotes evidence this discovery and appreciation: 

I saw correlations between his mood and focus level, and 
the use of the survey caused me to look at other fac- 
tors that played into focus and progress. The assessment 
format definitely helped me to know the correct way to 
conduct and improve my tutoring sessions. Through us- 
ing the survey, my awareness of student progress and the 
need to make adaptations became especially clear when I 
looked at the data on the Excel spreadsheet. I am not sure 
that without the reflection my sessions would have had 
gradual improvement over time. 

For instance. Student X struggled with complex opera- 
tions and fractions. Seeing this documented continuously 
reminded me to spend extra time in these areas. This was 
the most valuable part of having the written survey re- 
sults. As well, when days didn’t go well, I could look back 
and see that there were days that went well, and that was 
encouraging (and I could see what worked that day) . 

During the first half of the semester, I used the original 
tutoring survey yet felt that I was not evaluating anything 
directly pertaining to my sessions with Student Y. I later 
created a new survey to better track Student Y’s progress 
with measures of motivation, mood, and distractibility. I 
was better able to assess my effectiveness with this survey. 

Discussion 

Quantitative indices from participants about the appropriateness of 
the hardware and software for data collection in the field proved to be 
consistent across both study groups despite modifications implemented in 
the second year. Analysis of qualitative data also revealed that the iPAQ was 
not the most reliable method for capturing and storing field-based data. 
Several factors may have contributed to the less than positive responses 
from the participants regarding classroom-based data collection with 


handheld computers. Primarily, problems with the hardware provided 
challenges for the majority of the participants. Unreliable hardware (i.e., 
keyboards not connecting, failure to hold a charge) was most likely as- 
sociated with the capabilities and age of the handheld computing tools. 
Another contributing factor was the prior knowledge of handheld comput- 
ing tool use. Most of the participants experience with handheld devices 
was limited to cell phones, and only 10 of the 31 participants had ever 
used handheld computers before. Of the 31 participants in both years 
of the study, only one indicated using the cell phone to connect to the 
Internet. This lack of experience with connecting handhelds to comput- 
ers to transfer data could affect maintenance of handheld computers for 
retaining files. Loss of data was experienced by more than one participant 
because the handhelds lose all of the stored data when they lose charge. 

Unfamiliarity with software used to create the assessment instruments 
was also a potential reason for negative responses to questions regarding 
the appropriateness of the hardware and software tools for classroom-based 
formative assessment. Accurate and efficient data collection might have 
been a source of stress for the participants because the results were to be 
used to complete course assignments. 

Based on the quantitative and qualitative analysis of participant re- 
sponses, the researchers decided that further integration of handheld data 
collection would require more modern and reliable tools. 

Recognition of the value of technology to support formative assessment 
was consistent. Even though the hardware was determined to be a draw- 
back, the process of situating formative assessment within a field-based 
action research process scaffolds candidates’ understanding of the value of 
technology for formative assessment. Participants in both groups agreed 
that their understanding of how technology can be used for formative 
assessment was enhanced by this experience. 

Open-ended responses and reflection about the value of data-collection 
tools for formative assessment were, in general, very positive. The par- 
ticipants actively sought to reflect upon and transform their practices as 
well as their data-collection instruments based on trends they noticed in 
their work with tutees. 

Implications 

Although the quantitative responses indicate a generally neutral at- 
titude toward the use of handhelds for formative assessment, the qualita- 
tive responses indicate that the process of data collection did help the 
participants recognize the value of ongoing formative data collection. The 
lack of more vehement negative responses about the use of the iPAQ as a 
data-collection instrument is surprising in light of the chronic hardware 
challenges faced by both study groups. A variety of negative conditions 
unrelated to action research, formative assessment, survey creation, or data 
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Table 2: Content Analysis Themes Regarding Usefulness of Handheld for Collection of Formative Assessment Data 


Hardware Deficiencies 


Software & Instrumentation 
Changes 


Generally Useful 


"I had a really hard time keeping the IPAQ charged. It was either over-charged and lost it's data or the battery died when it wasn't 
charged enough. Maybe if I was using it on regular basis, ratber than just for this class I wouldn't have had this issue??" 

"I began by entering tbe information as soon as I was done by answering tbe survey questions. I lost everything during spring break 
because I didn't charge it. Instead of re-creating the survey, I just used Excel. I liked the layout better anyway." 

"The IPAQ is a great incorporation of technology, but the negatives far outweigh the positives. The negatives were that the IPAQ died 
and the information would be lost (if not charged, if charged too much)." 

"Because the IPAQ crashed, I created an Excel spreadsheet to track progress. The trends from the spreadsheet show an increase in 
the enjoyment of reading, and a decrease in verbal stimulation... Using the general survey too was too general, I liked creating one that 
was specific to my tutee." 

"I like the Data-ln-Hand software and it's capabilities; however, data entry for me was too tedious because of the open-ended questions 
I made. Entering long answers interrupted instructional time." 

"I would have been more successful if I had used a survey that measured more specific parts of progress made. The specific questions 
I did have were very helpful. Maybe if we had more training in howto write our own surveys that would help." 

"With some changes, the handheld could be very beneficial. The use of technology can speed along processes in the classroom - 
especially assessment! We were able to quickly and easily input data on our student. I can see how handhelds would be great because 
they are small and easily accessible." 

"The additional comments section was the most helpful (using the handheld survey tool). This is where I would record things that I 
wanted to remember to do next time, or things that I wanted to remember about the current session. I liked having these details." 

"I appreciated it though because it was a great thing to utilize to get myself thinking about the overall experience with Student X. I think 
I would recommend some future product that could also readily be incorporated into everyday life. If this were the case, it would also 
be useful to organize thoughts, lessons, activities, along with collecting data on student progress." 


collection must be considered as contributing to the less than enthusiastic 
responses from the participants about the handheld technology. Positive 
results that weren’t captured in the responses were the ease of use of the 
survey-creation tool and the simplicity of capturing and exporting results 
of data collection to the computer. The respondents’ acknowledgment 
that their understanding of classroom technology use was enhanced by 
this experience is also positive and encourages further research in this 
area. Situating action research within a one-to-one tutoring experience 
provides the scaffolding necessary to promote the development of forma- 
tive assessment skills and dispositions. 

Because of the limitations of the handheld computers used for this 
project and the advent of more reliable handheld technologies, future 
studies should utilize Smartphone technology and, as possible, integrate 
it in coursework for more than one semester and in more intensive and 
regular field experiences. Preservice teachers should be comfortable with 
the devices to remove the stress of participating in graded action research 
projects in field experiences, or research should be limited to participants 
who are comfortable with the tools. It is more likely that candidates will 
enter education courses with more prior experience due entirely to the 
advancement of cell phone technology. Formative assessment systems 
that generate information about student achievement are critical to 
ensuring the success of students in PK-12 classrooms, and field-based 
action research provides a context to facilitate this type of instruction. 
Handheld computing devices are inexpensive, multipurpose, and relatively 
ubiquitous. Future teachers need to learn how to use these tools effectively 
in their 2P'-century classrooms to measure, analyze, and communicate 
student progress to improve learning as a whole. 

Teacher educators, preservice and inservice, should recognize that 
preparing teachers to conduct ongoing formative assessment that informs 
instructional decision making is a critical part of instructional design. 
Embedding such experiences within an action research model and con- 
necting them to field-based data collection is clearly a viable method for 
scaffolding the development of skills and attitudes regarding formative 
assessment techniques and benefits. This model is theoretically sound and 
viable for teachers of all experience levels. 
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Appendix 

Postexperience Survey 

Questions regarding use and value of handheld tools for field-based data 
collection: 

1 . The handheld is a useful tool for collecting data in the field. 

2. DataInHand software made it easier to capture data in the 
field. 

3. The handheld and DataInHand facilitated completion of course 
assignments. 

4. The handheld and DataInHand allowed me to collect and 
process field observations in a timely manner. 

5. Using a handheld device and a consistent data-collection tool 
facilitated data analysis. 

6. Using these tools to collect data in the field was worthwhile. 

7. My understanding of how to use technology in the classroom 
has been enhanced by this experience. 
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